
JC16 Rec'd PCT7PT0 SEP 2 7 2001 

Practitioner's Docket No. U 013655-0 



Optional Customer No. Bar Code 




00140 

PATENT TRADEMARK OFFICE 



CHAPTER II 

TRANSMITTAL LETTER 
TO THE UNITED STATES ELECTED OFFICE (EO/US) 

(ENTRY INTO U.S. NATIONAL PHASE UNDER CHAPTER II) 

30 MARCH 1999 

PCT/IL00/002O1 30 MARCH 2000 30 NOVEMBER 1999 

INTERNATIONAL APPLICATION NO. INTERNATIONAL FILING DATE PRIORITY DATE CLAIMED 

A METHOD AND SYSTEM FOR SUPER RESOLUTION 

TITLE OF INVENTION ~ " 

David MENDLOVIC and Zeev ZALEVSKY 

APPLICANTS) ~ 
Box PCT 

Assistant Commissioner for Patents 
Washington D.C. 20231 

ATTENTION: EO/US 

NOTE: The completion of those filing requirements that can be made at a time later than 30 months from the priority date 
results from the Commissioner exercising his judgment under the authority granted under 35 USC 371(d). The filing 

CERTIFICATION UNDER 37 C.F.R. 1.10* 

(Express Mail label number is mandatory.) 
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I hereby certify that this correspondence and the documents referred to as attached therein are being deposited with the 
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Addressee," Mailing Label Number EL728245245US , addressed to the: Assistant Commissioner for Patents 

Washington, D.C. 20231. 
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WARNING: Certificate of mailing (first class) or facsimile transmission procedures of 37 C.F.R. 1.8 cannot be used 
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^WARNING; Each paper or fee filed by "Express Mail" must have the number of the "Express Mail" mailing label 
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"Since the filing of correspondence under § 1 10 without the Express Mail mailing label thereon is an 
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receipt will show the actual date of receipt of the last item completing the entry into the national phase. See 37 CF.R. 
§1.491 which states: "An international application enters the national state when the applicant has filed the 
documents and fees required by 35 USC 371(c) within the periods set forth in § 1.494 and § 1.495. " 

WARNING: Where the items are those which can be submitted to complete the entry of the international application into 

the national phase are subsequent to 30 months from the priority date the application is still considered to 
be in the international state and if mailing procedures are utilized to obtain a date the express mail 
procedure of 37 CF.R §L10 must be used (since international application papers are not covered by an 
ordinary certificate of mailing - See 37 C F R. §1.8. 

NOTE: Documents and fees must be clearly identified as a submission to enter the national state under 35 USC 3 71 otherwise 
the submission will be considered as being made under 35 USC 111. 37 CF.R. § 1.494(f). 



1 . Applicant herewith submits to the United States Elected Office (EO/US) the following items 
under 35 U.S.C. 371: 

a. [X] This express request to imm ed iately begin national examination procedures (3 5 

U.S.C. 371(f))- 

b. [X] The U.S. National Fee (35 U.S.C. 371(c)(1)) andotherfees (37 CF.R. § L492) 

as indicated below: 
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2.Fees 



CLAIMS 
FEE 


(1) FOR 


(2) NUMBER 
FILED 


(3) NUMBER 
EXTRA 


(4) RATE 


(5) CALCULA- 
TIONS 


[ ]* 


TOTAL 
CLAIMS 


64- 20 - 


44 


x $18.00 = 


$NOT PAID 




INDEPENDENT 
CLAIMS 


4-3 = 


1 


x $80.00 = 


NOT PAID 




MULTIPLE DEPENDENT CLAIM(S) (if applicable) + $270.00 




BASIC FEE** 


[ ] U.S. PTO WAS INTERNATIONAL PRELIMINARY EXAMINATION 
AUTHORITY 

Where an International preliminary examination fee as set forth in § 
1.482 has been paid on the international application to the U.S. PTO: 
[ ] and the international preliminary examination report states that 
the criteria of novelty, inventive step (non-obviousness) and 
industrial activity, as defined in PCT Article 33(2) to (4) have 
been satisfied for al] the claims presented in the application 
entering the national stage (37 CFR 1.492(a)(4)) $100.00 
[ ] and the above requirements are not met (37 CFR 1 .492(a)(1)) 
$690.00 






r X 1 US PTO WAS NOT TNTFRN ATTONAT PR FT TMTNARY 
EXAMINATION AUTHORITY 

Where no international preliminary examination fee as set forth 
in § 1.482 has been paid to the U S. PTO, and payment of an 
international search fee as set forth in § 1.445(a)(2) to the U.S. 
PTO: 

[ ] has been paid (37 CFR 1 .492(a)(2)) $710.00 

[ ] has not been paid (37 CFR 1.492(a)(3)) $1,000.00 

[ X ] where a search report on the international application has been 
prepared by the European Patent Office or the Japanese Patent 
Office (37 CFR 1.492(a)(5)) $860.00 






Total of above Calculations 


=$860.00 


SMALL 
ENTITY 


Reduction by l A for filing by small entity, if applicable. Statement may also be 
filed, (note 37 CFR 1.9, 1.27, 1.28) 






Subtotal 


$860.00 




Total National Fee 


$860.00 




Fee for recording the enclosed assignment document $40.00 (37 CFR 1.21(h)). 
(See Item 13 below). See attached "ASSIGNMENT COVER SHEET". 




TOTAL 


Total Fees enclosed 


$860.00 



*See attached Preliminary Amendment Reducing the Number of Claims. 
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[ X ] A check in the amount of $860.00 to cover the above fees is enclosed. 

[ ] Please charge Account Mo. in the amount of $ 

A duplicate copy of this sheet is enclosed. 



** WARNING: "To avoid abandonment of the application the applicant shall ^irnish to the United States Patent and 
Trademark Office not later than the expiration of 30 months from the priority date: * * * (2) the basic 
national fee (see § 1.492(a)). The 30-month time limit may not be extended. "37 C.F.R. § 1.495(b). 



WARNING: If the translation of the international application and/or the oath or declaration have not been 

submitted by the applicant within thirty (30) months from the priority date, such requirements may be 
met within a time period set by the Offfce 37 C.F.R. § 1.495(b)(2). The payment of the surcharge set 
forth in § 1.492(e) is required as a condition for accepting the oath or declaration later than thirty (30) 
months after the priority date. The payment of the processing fee set forth in § 1.492(f) is required for 
acceptance of an English translation later than thirty (30) months after the priority date. Failure to 
comply with these requirements will result in abandonment of the application. The provisions of § 1.136 
apply to the period which is set. Notice oj Jan. 3, 1993, 1147 O.G. 29 to 40. 



3. [ X ] A copy of the International application as filed (35 U.S.C. 371(c)(2)): 



NOTE: Section 1.495 (b) was amended to require that the basic national fee and a copy of the international application 
must be filed with the Office by 30 months from the priority date to avoid abandonment "The International 
Bureau normally provides the copy of the international application to the Office in accordance with PCT Article 
20. At the same time, the International Bureau notifies applicant of the communication to the Office. In 
accordance with PCT Rule 47. 1, that notice shall be accepted by all designated offices as conclusive evidence that 
the communication has duly taken place. Thus, if the applicant desires to enter the national stage, the applicant 
normally need only check to be sure the notice from the International Bureau has been received and then pay the 
basic national fee by 30 months from the priority date " Notice of Jan. 7, 1993, 1147 O.G. 29 to 40, at 35-36. See 
item 14c below. 



a. 


t ] 


is transmitted herewith. 


b. 


t j 


is not required, as the application was filed with the United States Receiving 






Office. 


c. 


[X] 


has been transmitted 




i. 


[ X ] by the International Bureau. 



Date of mailing of the application (from form PCT/IB/308): 

ii. [ ] by applicant on . 

Date 



4. [ X ] A translation of the International application into the English language (35 U.S.C. 
371(c)(2)): 

a. [ ] is transmitted herewith. 

b. [ X ] is not required as the application was filed in English. 

c. [ ] was previously transmitted by applicant on . 

Date 

d. [ ] will follow. 
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[X ] Amendments to the claims of the International application under PCT Article 19 (35 
U.S.C. 371(c)(3)): 

The Notice of January 7, 1993 points out that 37 C.F R § 1.495(a) was amended to clarify the existing and 
continuing practice that PCT Article 19 amendments must be submitted by 30 months from the priority date and 
this deadline may not be extended. The Notice further advises that: "The failure to do so will not result in loss of 
the subject matter of the PCT Article 19 amendments Applicant may submit that subject matter in a preliminary 
amendment filed under section 1.121. In many cases, filing an amendment under section L 121 is preferable since 
grammatical or idiomatic errors may be corrected. " 1147 O.G. 29-40, at 36. 



a. [ ] are transmitted herewith. 

b. [ ] have been transmitted 

i. [ ] by the International Bureau. 

Date of mailing of the amendment (from form PCT/IB/308): . 

ii. [ ] by applicant on . 

Date 

c. [ X ] have not been transmitted as 

i- [ X ] applicant chose not to make amendments under PCT Article 19. 
Date of mailing of Search Report (from form PCT/ISA/210): 24 JULY 2000 . 

ii- [ ] the time limit for the submission of amendments has not yet expired. 

The amendments or a statement that amendments have not been 
made will be transmitted before the expiration of the time limit under 
PCT Rule 46.1. 



[ X ] A translation of the amendments to the claims under PCT Article 19 (3 8 U S C 

371(c)(3)): 
a. [ ] is transmitted herewith. 

b- [ ] is not required as the amendments were made in the English language, 
c [ X ] has not been transmitted for reasons indicated at point 5(c) above. 

[X ] A copy of the international examination report (PCT/IPEA/409) 
[ X ] is transmitted herewith. 

[ ] is not required as the application was filed with the United States Receiving 
Office. 



[ ] Annex(es) to the international preliminary examination report 

a. [ ] is/are transmitted herewith. 

b- [ ] is/are not required as the application was filed with the United States 
Receiving Office. 

[ ] A translation of the annexes to the international preliminary examination report 

a. [ ] is transmitted herewith. 

b. [ ] is not required as the annexes are in the English language. 
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10. [ X ] An oath or declaration of the inventor (35 U.S.C. 371(c)(4)) complying with 35 
U.S.C. 115 

a. [ ] was previously submitted by applicant on . 



Date 

b. [ ] is submitted herewith, and such oath or declaration 
i. [ ] is attached to the application. 

ii- [ ] identifies the application and any amendments under PCT Article 19 
that were transmitted as stated in points 3(b) or 3(c) and 5(b); and 
states that they were reviewed by the inventor as required by 37 
C.F.R. 1.70. 

c. [ X ] will follow. 



Other document(s) or information included: 



[ X ] An International Search Report (PCT/ISA/210) or Declaration under PCT Article 
17(2)(a): 



a. 


[X 


] is transmitted herewith. 


b. 


[ ] 


has been transmitted by the International Bureau. 






Date of mailing (from form PCT/IB/308): 


c. 


[ ] 


is not required, as the application was searched by the United States 






International Searching Authority. 


d. 


[ ] 


will be transmitted promptly upon request. 


e. 


[ ] 


has been submitted by applicant on 






Date 


[X ] 


An 


Information Disclosure Statement under 37 C.F.R. 1.97 and 1.98: 


a. 


[ ] 


is transmitted herewith. 






Also transmitted herewith is/are: 




[ ] 


Form PTO-1449 (PTO/SB/08A and 08B). 




[ ] 


Copies of citations listed. 



b. [ X ] will be transmitted within THREE MONTHS of the date of submission of 

requirements under 35 U.S.C. 371(c). 
c [ ] was previously submitted by applicant on . 

Date 



13. [ ] An assignment document is transmitted herewith for recording. 

A separate [ ] "COVER SHEET FOR ASSIGNMENT (DOCUMENT) ACCOMPANYING 
NEW PATENT APPLICATION" or [ ] FORM PTO 1595 is also attached. 
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14. [X ] Additional documents: 

a. [ ] Copy of request (PCT/RO/1 01) 

b. [ X ] International Publication No. WO 00/59206 

i. [ X ] Specification, claims and drawing 

ii. [ ] Front page only 

c. [ ] Preliminary amendment (37 C.F.R. § 1.121) 

d. [ ] Other 



15. [ X ] The above checked items are being transmitted 

a. [ X ] before 30 months from any claimed priority date. 

b. [ ] after 30 months. 

16. [ ] Certain requirements under 35U.S.C. 371 were previously submitted by the 

applicant on 

? namely: 



AUTHORIZATION TO CHARGE ADDITIONAL FEES 



WARNING: Accurately count claims, especially multiple dependent claims, to avoid unexpected high charges if extra 

claims are authorized. 



NOTE: "A written request may be submitted in an application that is an authorization to treat any concurrent or future 
reply, requiring a petition for an extension of time under this paragraph for its timely submission, as 
incorporating a petition for extension of time for the appropriate length of time. An authorization to charge all 
required fees, fees under §1.17, or all required extension of time fees will be treated as a constructive petition for 
an extension of time in any concurrent or future reply requiring a petition for an extension of time under this 
paragraph for its timely submission. Submission of the fee set forth in § 1.1 7(a) will also be treated as a 
constructive petition for an extension of time in any concurrent reply requiring a petition for an extension of time 
under this paragraph for its timely submission. " 37 C F R § LI 36(a)(3). 

NOTE: "Amounts of twenty-five dollars or less will not be returned unless specifically requested within a reasonable time, 
nor will the payer be notified of such amounts; amounts over twenty-five dollars may be returned by check or, if 
requested, by credit to a deposit account " 37 C F R § 1 26(a). 



[X] The Commissioner is hereby authorized to charge the following additional fees that 
may be required by this paper and during the entire pendency of this application to 
Account No. 12-0425 



[X] 37 C.F.R. 1.492(a)(1), (2), (3), and (4) (filing fees) 

WARNING: Because failure to pay the national fee within 30 months without extension (37 C.F.R. § 1.495(b)(2)) 

results in abandonment of the application, it would be best to always check the above box. 

[ ] 37 C.F.R. 1.492(b), (c) and (d) (presentation of extra claims) 

NOTE: Because additional fees for excess or multiple dependent claims not paid on filing or on later presentation must 
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only be paid or these claims cancelled by amendment prior to the expiration of the time period set for response by 
the PTO in any notice of fee deficiency (37 C F.R § 1 492(d)), it might be best not to authorize the PTO to charge 
additional claim fees, except possible when dealing with amendments after final action. 

[X] 37 C.F.R. 1.17 (application processing fees) 
[X] 37 C.F.R. 1.17(a)(l)-(5)(extension fees pursuant to § 1.136(a). 
[X] 37 C.F.R. 1.18 (issue fee at or before mailing of Notice of Allowance, 
pursuant to 37 C.F.R. 1.311 (b)) 

NOTE: Where an authorization to charge the issue fee to a deposit account has been filed before the mailing of a Notice 
of Allowance, the issue fee will be automatically charged to the deposit account at the time of mailing the notice 
of allowance. 37 C.F.R. § 1.31 1(b). 

NOTE: 37 C.F.R. 1.28(b) requires "Notification of any change in loss of entitlement to small entity status must be filed in 
the application . . . prior to paying, or at the time of paying . . . issue fee. " From the wording of 37 C.F.R, § 
1.28(b): (a) notification of change of status must be made even if the fee is paid as "other than a small entity" and 
(b) no notification is required if the change is to another small entity. 

[ ] 37 C.F.R. § 1.492(e) and (f) (surcharge fees for filing the declaration and/or 
filing an English translation of an International Application later than 30 
months after the priority date). 



Reg. No.: 25,858 William R. Evans 




SIGNA^^fe^rfRACTITIONER 



(type or print name of practitioner) 

Tel. No.: ( 212) 708-1930 



P.O. Address 

Customer No.: 00140 



c/o Ladas & Parry 
26 West 61 st Street 
New York, N.Y. 10023 
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A Method and System for Super Resolution 



FIELD OF THE INVENTION 



The present invention relates to super resolution enhancement of 



sampled data. 



BACKGROUND OF THE INVENTION 

Super-resolution refers to the enhancement of resolutions for 
images of scenery or objects acquired by an image capture device, such as a 
scanner or a CCD video camera, beyond the resolution inherent in the capture 
device, and beyond other limitations of the capture device. Super-resolution is 
used primarily to enhance the quality of captured images, and to increase the 
information content therein. 

Many super-resolution techniques used with imaging sensors are 
based on micro scanning operations. When using micro-scanning operations, a 
scene or object being captured is sampled multiple times, each time with a sub 
pixel shift applied- 

Prior art techniques for super-resolution do not achieve exact 
super resolution, but rather they perform over-sampling. Exact super resolution is 
limited by the fill factor; Le., the ratio between the area of the region sensitive to 
radiation and the pitch. The pitch is the area between centers of adjacent detectors 
in a sensor panel - equivalently, the pitch is the area between adjacent pixels. For 
example, if the sensitivity to radiation only extends over half the distance 
separating detectors, then the fill factor is G.5 * 0.5 = 25%. With a 25% fill factor, 
exact super-resolution using prior art methods can be obtained up to a factor of 
two in resolution in each dimension, since the regions of sensitivity to radiation 
are only half the size of the pitch, and can thus be shifted by half-pixels in each 
dimension without overlapping. Additional super-resolution, beyond a factor of 
two, has been achieved by prior art methods at the expense of decreased contrast. 

For a fill factor of 80%, which is typical for image sensing 
devices, the limitation on exact super-resolution is ^Jq^^^*^' ^ e 

experimental barrier is approximately at a factor of 1.5, with some decrease in 
contrast. 
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Prior art methods for obtaining super resolution from captured 
images are based on estimating signal distortion. These methods perform optimal 
estimation of signal distortion from the captured images, using Bayesian 
techniques and using criteria such as Maximum Entropy. These methods are 
advantageous when image acquisition is made over a large distance (such as 
satellite data acquired from outer space), since atmospheric transmission and 
turbulence have a major impact on limiting the resolution obtained. 

One such method is described in L Cheeseman, B. Kanefsky, R. 
Kraft, J. Stutz, R. Hanson, "Super-Resolved Surface Reconstruction from 
Multiple Images/' in Maximum Entropy and Bayesian Methods, G. R. Heidbreder 
(ed.), Kluwer, the Netherlands, 1996, pgs. 293 - 308* This method is based on 
inverse graphics theory, and is used for ground modeling from outer space 
observations. An initial ground model is formed by letting each pixel "vote" on 
what the corresponding ground position should be, based upon the extent that the 
corresponding ground position contributes to that pixel. The initial ground model 
is then used to project what an image should be (Le., predict each pixel value). 
The differences between the predicted pixel values and the observed pixel values 
are used to update the ground model until it cannot be further improved. This 
procedure produces an increase in both spatial resolution and gray-scale 
resolution. 

Another such method is described in A. Zomet and S. Peleg, 
"Applying Super-Resolution to Panoramic Mosaics," IEEE Workshop on 
Applications of Computer Vision, Princeton, Oct 1998. This method attains super 
resolution using an iterative method for mosaicing* Given a video sequence 
scanning a static scene, a panoramic image can be constructed whose field of view 
is the entire scene. Each region in the panorama is covered by many overlapping 
image segments, and this can be exploited to enhance its resolution. 

Another such method is described in Pearson, T. L and Readhead, 
A. C S. "Image Formation by Self-Calibration in Radio Astronomy," Ann. Rev. 
Astron. Astrophys. 22, 1984, pgs. 97-130. This method uses non linear methods, 
and is based on optimal spectrum estimation. 
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SUMMARY OF THE INVENTION 



The present specification concerns data obtained by sampling a 
continuous signal, and describes methods and systems for enhancing the resolution 
of the data. For example, a barcode reader samples a barcode when scanning it, 
and the present invention can be used to enhance the resolution of the sampled 
data, thereby providing a better reconstruction of the barcode. For another 
example, a CCD camera samples an object or scene being viewed, to produce a 
digital image, and the present invention can be used to enhance the resolution of 
the digital image, thereby providing a better quality image. 

The present invention enhances sampled data by effectively 
decreasing the sampling period. When used with digital images, the present 
invention provides sub-pixel accuracy. An original image quantized into pixel 
area elements can be enhanced using the present invention to a finer granularity 
quantization with sub-pixel area elements. For example, the enhanced image can 
have one -eighth pixel granularity. 

The present invention overcomes the limitation of prior art 
methods to achieve exact resolution improvement by more than a factor of L5. 
Using the present invention, any desired resolution improvement may be obtained 
by attaching an appropriately designed mask to a sensor plane of a capture device. 

There is provided in accordance with a preferred embodiment of 
the present invention a method for enhancing the resolution of an image sensing 
device, including the steps of attaching a mask to a panel of detectors in an image 
sensing device, generating multiple fields of view, the multiple fields of view 
being related to one another by sub-pixel shifts, acquiring multiple images with 
the image sensing device from the multiple fields of view, and combining the 
multiple images into an enhanced image of higher pixel resolution than the pixel 
resolutions of the multiple images. 

There is further provided in accordance with a preferred 
embodiment of the present invention a system for enhancing the resolution of an 
image sensing device, including an image sensing device comprising a panel of 
detectors, a mask attached to said panel of detectors, a motion generator 
generating multiple fields of view, the multiple fields of view being related to one 
another by sub-pixel shifts, image acquisition circuitry housed within said image 
sensing device acquiring multiple images from the multiple fields of view, and a 
combiner combining the multiple images into an enhanced image of higher pixel 
resolution than the pixel resolutions of the multiple images. 

3 
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There is still further provided in accordance with a preferred 
embodiment of the present invention a method for enhancing the resolution of an 
image sensing device, including the steps of creating replicas of fields of view 
using an optical element attached to an image sensing device, acquiring multiple 
images with the sensing device from the replicas of fields of view, and combining 
the multiple images into an enhanced image of higher pixel resolution than the 
pixel resolutions of the multiple images. 

There is additionally provided in accordance with a preferred 
embodiment of the present invention a system for enhancing the resolution of an 
image sensing device, including an image sensing device, an optical element 
attached to the image sensing device, the optical element being such as to create 
repiicas of fields of view, image acquisition circuitry housed within the image 
sensing device acquiring multiple images from the replicas of fields of view, and a 
combiner combining the multiple images into an enhanced image of higher pixel 
resolution than the pixel resolutions of the multiple images. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



The present invention will be more fully understood and 
appreciated from the following detailed description taken in conjunction with the 
drawings in which: 

Figure' 1 is an illustration of a one-dimensional cross-section of a 
prior art image sensing device, such as a CCD camera or a scanner; 

Figure 2 is a prior art illustration of a pixel sensitivity 
(apodization) function, for modeling averaged light intensity arriving at a detector; 

Figures 3A - 3C are illustrations of the results of computer 
simulations of a preferred embodiment of the present invention, as applied to CCD 
cameras; 

Figures 4A - 4D are further illustrations of the results of 
computer simulations of a preferred embodiment of the present invention, as 
applied to CCD cameras; 

Figures 5A - 5C are illustrations of the results of computer 
simulations of a preferred embodiment of the present invention, as applied to 
barcode readers; 

Figure 6 is an illustration of three samplings of an object by a 
sensing device, separated by sub-pixel shifts, in accordance with a preferred 
embodiment of the present invention; 

Figure 7 is a simplified illustration of a mask attached to a sensor 
panel of array detectors in an image sensing device, in accordance with a preferred 
embodiment of the present invention; 

Figure 8 illustrates the shape of a mask in accordance with a 
preferred embodiment of the present invention; 

Figures 9A - 9J illustrate the use of a mask in obtaining super- 
resolution, in accordance with a preferred embodiment of the present invention; 

Figure 10 illustrates an enhanced image obtained by applying 
super-resolution to the low resolution CCD captured image illustrated in Figure 
3B, without the use of a mask; and 

Figure 11 is a simplified illustration of an alternating 
("checkerboard") mask attached to a sensor panel of array detectors in an image 
sensing device, in accordance with a preferred embodiment of the present 
invention. 
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DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT 



The present invention concerns digital images captured by digital 
image acquisition devices, such as digital cameras and scanners, and provides a 
method and system for obtaining super-resolution imagery. Super-resolution 
refers to the ability to obtain a resolution of a digital image that is greater than that 
native resolution of the image acquisition device. The present invention uses 
multiple low resolution captures of a picture to obtain a high resolution digital 
image. For example, if 64 images of an object are captured by a CCD camera 
having a spatial resolution of 32 x 32 pixels, the present invention can be used to 
integrate the captured data and provide a digital image of the object at a spatial 
resolution of 256 x 256 pixels. The present invention makes efficient use of 
captured image data, in that there is a direct one-to-one relationship between the 
number of images captured and the ratio of resolution enhancement. That is, if K 
images are captured then the present invention enhances resolution of the capture 
device by a factor of K. 

Additionally, the present invention applies to acquisition devices, 
regardless of their dynamic color range. In the example described above, the CCD 
camera can have as low as 1 bit per pixel color depth, or as high as 8 bits per pixel 
and higher, and the present invention is applicable. 

As used in the present specification in reference to digital images, 
the terms "low resolution" and "high resolution" are intended to be relative 
terms, used to indicate an advantage of the present invention in providing digital 
images of an input object at a higher resolution than those produced by an image 
acquisition device. 

Reference is now made to Figure 1, which illustrates a one- 
dimensional cross-section of a prior art image sensing device, such as a CCD 
camera or a scanner. An image sensing device 100 consists of an array of 
detectors 110 in a sensing panel (not shown), each of which measures a light 
intensity of a scene 120 at a specific pixel location. Light rays 130 emanating 
from scene 120 are diffracted by a lens 140, and arrive at detectors 110. 

The array of detectors 110 in the sensing panel is two- 
dimensional, and the one -dimensional array of detectors illustrated in Figure 1 is 
but a single cross-section of the two-dimensional array. Similarly, Figure 1 
illustrates a single cross-section of scene 120. The direction of the one- 
dimensional array of detectors is denoted by a variable x> and the light intensity of 
the scene is correspondingly denoted by u(x). The variable x is a continuous 
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variable, and scene 120 corresponds to a continuous signal. The array of detectors 
110 samples scene 120, and produces a discrete digital image, comprising pixel 
locations and pixel color intensities. The spacings between detectors corresponds 
to the pixel size for the sampled digital image, and is referred to as the "pitch." 

Reference is now made to Figure 2, which illustrates a one- 
dimensional cross-section of a prior art pixel sensitivity (apodization) function, for 
modeling averaged light intensity arriving at a detector. The measured light 
intensity at each detector 110 (Figure 1) is not a measure of the light intensity at a 
single point location of scene 120. Instead, it is an average intensity, averaged by 
a pixel sensitivity function g(x), or apodization. The pixel sensitivity function is 
illustrated as a one-dimensional function g(x) in Figure 2, but g(x) is only a single 
cross-section of a two-dimensional pixel sensitivity function g(x, y). 

The support of the two-dimensional pixel sensitivity function 
corresponds to the region that is sensitive to radiation. This region typically 
extends across an area that is contained within the area between neighboring pixel 
centers. The ratio between the area of the two-dimensional support of the pixel 
sensitivity function and the area between pixel centers containing it is referred to 
as the "fill factor" 

It is the averaging with g(x) that gives rise to complications in 
obtaining exact super-resolution in prior art methods. 

Reference is now made to Figures 3A - 3C, which illustrate the 
results of computer simulations of a preferred embodiment of the present 
invention, as applied to CCD cameras. Figure 3A shows an input picture with 
spatial resolution of 256 x 256 pixels. Figure 3B shows an image of the input 
picture, captured by a CCD camera having a spatial resolution of 32 x 32 pixels, 
and 8 bits per pixel color depth. Figure 3C shows the reconstructed image 
obtained by the present invention by using 64 pictures like the one illustrated in 
Figure 3B. It can be seen that substantially all of the spatial details contained 
within the input picture are reconstructed in the image of Figure 3C 

Reference is now made to Figures 4A - 4D, which further 
illustrate the results of computer simulations of a preferred embodiment of the 
present invention, as applied to CCD cameras. Figure 4A shows a low resolution 
image captured from the input picture illustrated in Figure 3A, using a CCD 
camera having a spatial resolution of 32 x 32 pixels and a color depth of 1 bit per 
pixel. It can be seen that the image shown in Figure 4 A is a black and white 
image. Figure 4B shows the reconstructed image obtained by the present 
invention by using 64 pictures like the one illustrated in Figure 4A. It can be seen 
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that although Figure 4B is not identical to Figure 3A, it nevertheless is a much 
better approximation than is Figure 4A, both in terms of spatial resolution and in 
terms of dynamic color range. 

Similarly, Figure 4C shows a low resolution image captured from 
the input picture illustrated in Figure 3A, using a CCD camera having a spatial 
resolution of 32 x 32 pixels and a color depth of 2 bits per pixel. It can be seen 
that the image shown in Figure 4C is comprised of four colors. Figure 4D shows 
the reconstructed image obtained by the present invention, by using 64 pictures 
like the one illustrated in Figure 4C It can be seen that Figure 4D is almost 
identical to Figure 3A. 

When the present invention was applied to a CCD camera having 
a spatial resolution of 32 x 32 pixels and a color depth of 4 bits per pixel (not 
shown), the reconstructed image (not shown) was indistinguishable from the 
original. 

Reference is now made to Figures 5 A - 5C, which illustrates the 
results of computer simulations of a preferred embodiment of the present 
invention, as applied to barcode readers. Shown in Figure 5A is an original 
barcode. Shown in Figure 5B is the image captured by a bar code reader. As a 
result of the limited resolving capability of the barcode reader, the captured image 
shown in Figure 5B is substantially different from the barcode shown in Figure 
5A. Shown in Figure 5C is a reconstructed image of the barcode using the present 
invention, based on eight vertical replications. A comparison between Figure 5A 
and Figure 5C indicates that the present invention successfully overcomes the 
limitations of conventional barcode readers in resolving scanned barcodes. 

The present specification describes the process for acquiring 
multiple low resolution images from a single picture, and the way to combine the 
multiple low resolution image data to generate a high resolution image. 

Conventions and Notation 

In the ensuing description it is assumed that a sensing device 
samples an object being acquired in lines, such as horizontal lines, using N pixels 
per line. The present invention is preferably applied separately in two dimensions, 
such as in a horizontal dimension and a vertical dimension, to increase the pixel 
resolutions in each of the dimensions. Since the applications of the present 
invention in each of the dimensions are similar, for the sake of clarity and 
defmiteness the present specification describes a preferred embodiment of the 
present invention in a single dimension, such as a horizontal dimension. In 
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particular, rather than use two coordinates (x, y) in the discussion below, a single x 
coordinate is used, and rather than work with double integrals and double sums, 
single integrals and single sums are used. Similarly, two-dimensional doubly 
periodic masking of a two-dimensional array of detectors in a sensing panel is 
described hereinbeiow using a one-dimensional periodic masking function m(x). 

When the sensing device samples a line, the detected energy of 
the n-th pixel is given by 

u[n] = ju(x + n&x)g(x)dx , n = 0, 1, N-l , (1) 

—00 

where u(x) denotes the captured object along a specific line, Ax denotes the pixel 
width and g(x) denotes the pixel sensitivity (i.e. apodization). 

Multiple Image Acquisition 

In a preferred embodiment of the present invention, the sensing 

device samples a line K times, each time with a shift in the captured object relative 

to the capture device by an amount of Ax / K. Letting ufcfnj denote the detected 

energy of the n-th pixel in the £-th sampling, Equation 1 generalizes to 

00 Ax 
u t [»] = J"(* + n Ax + k —)g(x)dx , 

n = 0,l, N-1; A = ft I, K-l . (2) 
Together, the various samples uy£n] provide KN samples given by 

y[n] = \u(x + n —)g{x)dx , n - ft I, KN-1 . (3) 

In an preferred embodiment of the present invention, the shifts in 
successive image acquisitions by Ax I K are implemented by using vibrations of 
the sensing device. For example, the sensing device can be placed on a vibrating 
platform. The parameters of the vibrations are estimated by vibration sensors or 
by use of appropriate algorithms. The vibration parameters are used to 
synchronize the sampling of the object so that the sampling occurs at times when 
the vibrated sensing device is located at appropriate sub-pixel shifts. 

In an alternate embodiment of the present invention, an assembly 
of one or more rotated mirrors can be used to shift successive fields of view of a 
sensing device by sub-pixel shifts. The rotated mirrors reflect the scene to the 
sensing device. 

In yet another embodiment, the shifts in successive image 
acquisitions by Ax I K can be implemented using an optical element instead of the 
imaging lens of the sensing device. The optical element is attached to the aperture 
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of the sensing device, and serves like a grating to create replicated views of an 
object. Typically gratings result in replicas sinc^ the Fourier transform of a 
grating is an impulse train of delta functions. This embodiment using an optical 
element to create replicated views is particularly well suited for objects that are 
sufficiently small so that the replicas do not overlap. 

The abovementioned optical element can be designed, for 
example, by a multi-facet lens. It can also be designed by an algorithm such as 
the one described in Z. Zalevsky, D. Mendlovic and A. W. Lohmann, Gerchberg- 
Saxton algorithm applied in the fractional Fourier or the Fresnei domain, Optical 
Letters 21 t 1996, pages 842 - 844, the contents of which are hereby incorporated 
by reference. 

It should be apparent to those skilled in the art that it is not 
necessary that the multiple acquired images be separated by identical sub-pixel 
shifts. The present invention applies to multiple acquired images separated by 
sub-pixel shifts of arbitrary sizes. Moreover, successive shifts between acquired 
images do not have to be sub-pixel. The shifts can be larger than one pixel shifts, 
as long as the multiple acquired images are all non-integral pixel shifts from one 
another. Specifically, the relative shifts of the multiple acquired images with 
reference to a fixed origin can be set to values r#dx, rjdx, ... , rgvj^x, as long as 
the differences rj~ rj are non-integral for any distinct indices i and /. The case of 
equal sub-pixel shifts described above corresponds to rfc ~ k I K. 

Reference is now made to Figure 6, which illustrates three 
samplings of an object by a sensing device, separated by sub-pixel shifts, in 
accordance with a preferred embodiment of the present invention. The object is 
indicated by a curve u(x), denoted by reference numeral 610* The curve u(x) 
denotes the color of the object along a line, as a function of position, x. Also 
indicated in Figure 6 is a curve g(x) denoting the sensitivity of a pixel, denoted by 
reference numeral 620. The curve g(x) is a characteristic of the sensing device, 
and indicates how color values are localized by averaging. 

Each row in Figure 6 indicates a sampling. The first row samples 
the object u(x) at positions indicated by dashed vertical lines 630. The six pixel 
values are denoted by uq[0], uoflj, ... uq[5]. The second row samples the object 
u(x) at shifted positions indicated by dashed vertical lines 640. The six pixel 
values are denoted by utfO], ujfl], ... uj{5]. The third row samples the object 
u(x) at shifted positions indicated by dashed vertical lines 650. The six pixel 
values are denoted by U2[0] f U2[2], ... ^2PJ- 



10 



WO 00/59206 PCT/ILOO/00201 

The samplings indicated by the second and third rows is often 
referred to as " sub-pixel 7 ' sampling, since these samplings are centered at 
fractional pixel locations. Specifically, in Figure 6, the samples in the second row 
are centered at the one-third pixel locations, and the samples in the third row are 
centered at the two-thirds pixel locations. The value of K for the geometry 
illustrated in Figure 6 is K = 5. The three sets of samples together form the 
eighteen samples yfO] = u 0 f0], yfl] = ujfO], yfl] = u 2 [0], y[3] = uq[1], 
y[17] - U2[5] f form the full set of samples at all of the pixel thirds locations. 
These eighteen samples yfn] are used to reconstruct the values u(0) 9 ), 

«(-—-), «(— 7" ) of the object u(x), 

3 3 

The present invention uses the samples yfnj to approximately 
Ax 

reconstruct the values of u(n — ). Determining these reconstructed values 

K 

achieves super-resolution with a resolution enhancement factor of K, since the 
sensing device has a resolution of N pixels per line, and the reconstructed signal 
has KN pixels per line. 

In order to describe the reconstruction of the values u(n — ) from 

the samples yfn], a classical result from signal processing is used, which relates 
the discrete Fourier transform of a sampled analog signal to the continuous 
Fourier transform of the analog signal. 

A Bit of HLst orv: Frequency Representation of Sampling 

In the ensuing description, s(t) is used to denote a continuous time 
signal, and S(jjQ) is used to denote its continuous-time Fourier transform, defined 
by: 

S(jQ)=[j(t)e^dt . (4) 

Periodic sampling of s(t) with a sampling period of T produces a discrete time 
signal sfn] y defined by: 

sfn] = s(nT) . (5) 
The discrete-time Fourier transform of sfn] is denoted by S(d m ), and is defined 
by: 

S(e>»)= X>[*K"" ■ ( 6 > 

An age-old result, referred to as the Poisson Summation Formula 
and which is one of the most prominent formulas used in discrete signal 

11 



WO 00/59206 PCT/ILOO/00201 

processing, provides a relationship between the continuous-time Fourier transform 
S(jf2) of a continuous time signal and the discrete-time Fourier transform S(el^) of 
the sampled signal. Specifically, the Poisson Summation Formula states that 

f>r) = i|; S (-;M). (7) 

In fact, more generally the Poisson Summation Formula establishes that 
^ s{ t + nT) = ±^S(-j?f-)e ! T , (8) 

of which Equation 7 above is a special case corresponding to t = 0. 

A simple proof of Equation 8, as indicated on page 210 of Rudin, 
W., " Real and Complex Analysis: Second Edition," McGraw-Hill, 1974, is as 
follows. The function f(t) defined by 

/(r)«2> + «r) (9) 

is a periodic function with period T. As such, it has a standard Fourier series 
expansion of the form 

where the Fourier coefficient cfc is given by 

c^j^me^dt. (11) 
Upon substituting Equation 9 into Equation 11 it can be seen that 

c 4 -^£^^-ls(-/^>, (12) 

from which Equation 8 follows when Equation 12 is substituted back into 
Equation 10. 

If the function s(t)e' 3 r is used in Equation 8, instead of the 
function s(c) 9 one obtains the result 



.ax 



± s(t + nT)e^ -LL± SU^-)/ 1 - . (13) 
By setting t = 0 in Equation 13 one arrives at the familiar sampling equation 

which appears as Equation 3.20 in A. V. Oppenheim and R. W. Schafer, 
"Discrete-Time Signal Processing," Prentice Hall, 1989. Equation 14 indicates 
that the discrete Fourier transform of a sampled analog signal is comprised of 
periodically repeated copies of the continuous Fourier transform of the analog 
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signal. The copies are shifted by integer multiples of the sampling frequency and 
superimposed. Equation 14 is often expressed in terms of delta functions; namely, 

that the Fourier transform of an impulse train ^S(t-nT) is itself an impulse 

tram — ) 5(Q. ) . 

It is noted that Equation 14 can also be derived directly from 

.ox 

Equation 7 by using s(t)e T instead of s(t). 

For values of co that are small enough to avoid significant aliasing 
in Equation 14, say co < Cl s , one can approximate the sums on the right hand sides 

of Equations 13 and 14 by the term with k = 0. When this approximation is made, 
it in turn leads to the approximation 

^s(t + nT)e- Jam ~S(e J ")e T , — <Q S . (15) 

It is noted that as the sampling period, T, decreases, the approximation in 
Equation 15 becomes more accurate. The increase in accuracy is due to the fact 
that the aliasing, or overlap, between the copies on the right hand sides of 

Equations 13 and 14 decreases, since the shifts — between copies increase. 

The present invention uses a version of Equation 15 to obtain 
super-resolution. Specifically, instead of sampling s[n] = s(nT) as in Equation 5, 
the sampling is done through an integral 

y[n]= ^s(t + nT)g(t)dt , (16) 

where g(t) represents the sensitivity, or apodization, of a pixel. Multiplying both 
sides of Equation 15 by g(t) and integrating, results in 



y(0*s(OC(-y~) > 



<fl,. (17) 



which is the form used in the present invention. 

Referring back to Equation 3, it follows from Equation 17, upon 
Ax 

replacing T by — , that 
K 



G(-j—-) 
Ax 



Ax 



<a, (18) 



Ax 

where U(el^) is the discrete Fourier transform of the samples u(n— ), sought to 

K 

be reconstructed; namely, 
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U(en =J^u(n—)e-^ . (19) 

The Fourier transform Y(eJ & ) is known from the samples y[n] in Equation 3, and 

coK 

the Fourier transform <?(-/ )is determined from knowledge of the sensitivity 

At 

function g(x). It is noted that as K increases, the approximation in Equation 18 
becomes more accurate. 

Regarding the Fourier transform Y(ei m ), the values of y[n] set 
forth in Equations above are only defined for indices n = 0, 1, KN-1. For 
indices n outside of this range, y[n] is preferably defined to be zero. With such a 
convention, the Fourier transform YfeJ* 0 ) is given by the finite sum 

W)= X^K^ • ( 2 °) 

In a preferred embodiment, the present invention uses Equation 
18 to determine the Fourier transform U(eJ**) 7 and then reconstructs the values 

Ay 

u(n — ),n = 0,l,...,KN-l, from U(d a ) by using an inverse discrete Fourier 
K 

transform. Preferably a Fast Fourier Transform (FFT), as is familiar to those 
skilled in the art, is used to invert the spectral values U(el m ). The reconstruction is 
approximate, since Equation 18 is itself only an approximation. Since it is 
necessary to reconstruct KN values of u(x), it is accordingly necessary to apply 

Equation 18 at KN frequencies that are spaced apart one from the next, since 

KN 

the Fourier transform U(ef*) is periodic with period 2rt. In a preferred 
embodiment of the present invention, Equation 18 is applied at the KN frequencies 

a> = — , k = Q,l,...,KN-l. (21) 
KN 

When the Fourier transform U(e3") is inverted it is possible that 
Ax 

values of u(n — ) are non-zero outside of the index range n = 0, 2, KN~L As 
K 

it is only the values of u(n—) within the index range n = 0, 1, KN- 1 that are 

K 

of interest, the values outside this index range can be ignored. 

An apparent difficulty arises in implementing Equation 18 if there 

are frequencies & as given in Equation 21, at which G(-j )-0, since this 

Ax 

would entail division by zero at such frequencies. 

The function G(-]D) is typically a real-valued function, since the 
pixel sensitivity function g(x) is typically symmetric about x = 0. Since a pixel is 
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typically more sensitive to light in its central area and less sensitive in its outer 
regions, the pixel sensitivity function g(x) is typically "bell-shaped," similar to 
Gaussian functions. In case g(x) happens to be a Gaussian function, its Fourier 
transform has no zeros, and there no divisions by zero are encountered in 
implementing Equation 18. However, for other sensitivity functions, such a 
difficulty can arise. 

The present invention avoids such a difficulty by using specially 
constructed masks, as described hereinbelow. 



Masking 

In order to avoid division by zero in Equation 18, the present 

invention preferablv emplovs an optical device to remove zeros of G(—j ) that 

Ax 

are less than the frequency Q s . The frequency D s is the maximal frequency in the 
reconstructed data. 

The present invention preferably attaches a mask in the form of a 
fine transmission grating with a period of Ax to the sensing device panel. As 
mentioned hereinabove, the mask under discussion is really a two-dimensional 
doubly periodic mask that covers a two-dimensional array of detectors. For the 
sake of clarity and definiteness it is being described as a single dimensional 
periodic mask, with the understanding that the one-dimensional analysis presented 
herein applies to each of the two dimensions of the sensing device panel. 

Reference is now made to Figure 7, which is a simplified 
illustration of a mask attached to a sensor panel of array detectors in an image 
sensing device, in accordance with a preferred embodiment of the present 
invention. A sensor panel 700 in an image sensing device contains a two- 
dimensional array of detectors 710. Detectors 710 are depicted as individual cells 
within sensor panel 700. The array of detectors is periodic, with periodic spacings 
Ax and Ay in the horizontal and vertical dimensions, respectively* The spacings Ax 
and Ay correspond to the pixel spacings in the sampled digital image obtained 
from the image sensing device. A masking pattern 720 in the form of a fine sub- 
pixel transmission grating is used to generate a doubly periodic mask, with 
respective periods Ax and Ay in the horizontal and vertical dimensions. In a 
preferred embodiment of the present invention, the doubly periodic mask is 
attached to sensor panel 700. Each detector 710 is masked by the same masking 
pattern 720. 
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Use of a mask decreases the effective size of each pixel so that the 
Fourier transform GQQ) of its sensitivity function has a wider band, and its zeros 
accordingly move to higher frequencies. The apodized pixel, determined by the 
transmission pattern of the grating, is preferably designed in an iterative manner, 
as described hereinbelow, so that the bandwidth of the Fourier transform GQD) of 
its sensitivity function is wider than the maximal frequency to be reconstructed. 
Use of a transmission grating has the disadvantage of decreasing the energy sensed 
at each pixel, since it blocks out some of the illumination. 

A mask is described mathematically by a spatial function m(x), 
preferably taking values 0 and 1. The effect of a mask is to modify the pixel 
sensitivity function from g(x) to p(x) = g(x)m(x). An objective of choosing a 
suitable mask is to obtain a function p(x) whose Fourier transform does not have 
zeroes at frequencies that are between 0 and so that when the Fourier 

transform P(-j ) of the modified pixel sensitivity function p(x) is used in 

Ax 

Equation 18 instead of G(- / ) , there is no division by zero. 

Ax 

A rationale for using a mask to avoid zeroes of G is that typically 
an effect of a mask is to add frequencies that may have not been present 
beforehand. 

To construct a mask as desired, the present invention preferably 
uses a grating in the form of a plurality of optically transparent slits. 

m(x) = l-frect(-^)®S(x-x i ) , (22) 

where M is the total number of slits, xj, x% . . xm & e locations of the slits and 
5xi is the width of the z-th slit. The function rectQ is given by 

fX -1/2<z<1/2, 
rect(z) = l i . (23) 

[0, otherwise 

and ® denotes convolution. 

Reference is now made to Figure 8, which illustrates the shape of 
a mask in accordance with Equation 22. Such a mask consists of M slits 810, 
centered at positions xj, x% each having respective widths of 

Preferably /the value of M is a number between zero and K> and the values &/ are 
each approximately equal to the sub-pixel size 4c/£ As can be seen in Figure 8, 
the values of the mask m(x) are 0 inside the slits and 1 outside the slits. The total 
width of the mask if Ax; namely, the width of a pixel. The mask is intended to 
affect the shape of g(x) at sub-pixel granularity. In practice, the granularity is 
discretized to Ax I K; namely, the sub-pixel shift between successive images 
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acquired by the image sensing device. For example, referring back to Figure 3C, 
the mask used in reconstructing the image shown therein is [0 1 0 1 1 0 0 0], each 
of the eight entries in this array representing a sub-pixel size of one eighth of a 
pixel. Thus the function m(x) for the mask used in Figure 3C is given for 
0 < x < A,r by 

Wv\-/ 1, t:Ax<*<-Ax, or -Ax<x <-Ax /nA , 
m W= n 8 8 8 8 (24) 



otherwise 

The sensitivity function of each masked pixel is given by 
p(x) = g(x)m(x) , (25) 
and it can be seen that the Fourier transform of p(x) is given by 
P(]Q) = P(jQ;x x ,x 2 ,„. y Xy ;<&i,& 2 ) = 

= G(jn) - £ ® [e"** sinc(^Q^)] , (26) 

where sinc(z) denotes the function sinc(z) = Smz . 

z 

Preferably, to determine an optimal mask, a predetermined value 
of M is selected, and a search is made for position values xj, X2, xm and 
widths &j, 3x2, Sxjrf for which the is bounded away from zero as 

much as possible. Specifically, define 
<j{x x y x z ,.„, x iVf \Sx ly Sx 2 Sx M ) = 

= rrin\PU&iXi > *i ^ ; &l y &z )\ > ( 27 > 

where i2 5 denotes the frequency up to which no significant aliasing occurs, as 
defined above. Then optimal values for xj, X2, x^f and Sxj, Sx% are 
preferably determined by 

, JCo , , ?*•*? ) 

^arg-maxcr^,^,...,^;^,^,,...,^) , (28) 
where arg-max denotes the arguments xj, jc9, *M and Sxj, 3x2, — > &flf which 
maximize the function cr. 

The search range over which optimal values of xj, X2, x${ and 
Sxi, Sx2, 3xm are sought is such that the total 3xj + 3x2 + »• + <&Af of a ^ the 
widths does not exceed the pixel width Ax. Additionally, each individual width 
Sx' t must not be less than 2A, where X is the wavelength of the incoming light - 
otherwise, the light will not be affected by the mask. Moreover, construction of a 
mask and attaching it to the detectors may further limit the widths &/ to exceed a 
minimum value of Sx m i n . 

From energy considerations, it is clear that on the one hand, it is 
desirable that each width Sx[ be as small as possible, since &/ is proportional to 
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the energy blocked by the mask. On the other hand, this must be traded off against 
the condition that [P(/Q)| be bounded away from zero. 

Specifically, if Et denotes the total light energy for a typical 
viewing target, when viewed without a mask, then the energy detected when a 
mask is present is given by 

£-£,(1--^— ) , (29) 
Ax 

since the energy blocked by the slits of the mask is proportional to the total width 
of the slits combined. 

Thus for a given minimum energy E m i n: required to activate the detectors, the 
widths Sxi are constrained to satisfy 

£& £ . <(l-^)Ax . (30) 

Condition (30), together with the condition that Sx i > &c min , for i - 1, 2, M, 
determine the search range for widths &?> — > over which a maximum of 
ofxj, x2j ...» *M> foh &2> •••» 8*M) 1S sought. 

When values of xj, xj, xm mci 3xj, 8*2* — » ^ 
determined according to Equation 28, Equation 22 is used to generate an optimal 
mask. 

In and alternative embodiment of the present invention, rather 
than maximize ofxj, r?, xm; &j, 3x2, —> & e v ^es of xj, x% Xji/ 
and 5x j, dxz **> ^ determined by maximizing a weighted function a(xi, x% 
x^/; &2, &3 Sxm) w (3xj, 3x2, ^AfA where w is an appropriate weight 
function. For example, w can put higher weight on smaller values of the widths 
5c/, so that preference is given to masks having slits with smaller widths. 

Reference is now made to Figures 9A - 9J, which illustrate the 
use of a mask in obtaining super-resolution, in accordance with a preferred 
embodiment of the present invention. Figure 9A illustrates an input object, such 
as a barcode, represented as a function u(x). Figure 9B illustrates the Fourier 
transform of u(x). Figure 9C illustrates a pixel sensitivity function g(x). Figure 
9D illustrates the Fourier transform ofg(x). 

Figure 9E illustrates conventional sampling of u(x) as a digital 
signal, without using super resolution enhancement. Figure 9F illustrates the 
Fourier transform of the digital signal of Figure 9E. Figure 9G illustrates 
sampling of u(x) as a digital signal with a resolution eight times higher than the 
sampling illustrated in Figure 9E. Figure 9H illustrates the Fourier transform of 
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the digital signal of Figure 9G. Figure 91 illustrates a mask used in accordance 
with a preferred embodiment of the present invention. 

Figure 9J illustrates the Fourier transform of a masked pixel, 
using the mask of Figure 9L By comparing Figure 9D with Figure 9J it can be 
seen that the spectrum of the masked pixel does not contain zeros within the 
region displayed. It can further be seen that an effect of the masking is to expand 
the spectrum of g(x). 

Reference is now made to Figure 10, which illustrates an 
enhanced image obtained by applying super-resolution to the low resolution CCD 
captured image illustrated in Figure 3B, without the use of a mask. As can be seen 
in Figure 10, artifacts in the form of vertically oriented white stripes arise due to 

the presence of zeroes of G(- ; — ) . When a mask is applied in accordance with 

Ax 

a preferred embodiment of the present invention, the zeroes of G(-j ) are 

Ax 

removed, and the result obtained is the image shown in Figure 3C It may thus be 
appreciated that the use of a mask serves to remove the artifacts indicated in 
Figure 10. 

Improved Approximation 

As explained hereinabove, the approximation in Equation 18 is 
based on the assumption that a is small enough to avoid aliasing in Equation 14. 
A more accurate approximation can be made by taking into consideration the 
aliasing between adjacent copies of S in the right-hand side of Equation 14. 
However, in order to obtain the increase in accuracy, it is necessary to increase the 
sampling rate. 

It is appreciated by those skilled in the art that the assumption that 
aliasing between copies of S in Equation 14 is due to overlap between adjacent 

terms and S{j^^-)vs equivalent to the assumption that reduction of 

the sampling period from T to T/2 removes aliasing. Moreover, the information at 
frequency co in the discrete Fourier transform of a signal sampled with sampling 
period T corresponds with the information at both frequencies &/2 and coll + ;rin 
the discrete Fourier transform of the signal sampled with sampling period 7/2. A 
precise formulation of this relationship appears in the above referenced 
Oppenheim and Schafer as Equation 3.76, which describes a frequency-domain 
relationship between the input and the output of a sampling rate compressor 
( u downsampiing"). 
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These observations above are a basis for an improved 
approximation used in accordance with a preferred embodiment of the present 
invention, as described hereinbelow. Specifically, replace the sampling period, T, 
by T/2 in Equation 16: 

y[n]=^s(t + n^)g(t)dt . (31) 
Under the assumption that aliasing in Equation 14 is due to overlap between 
adjacent terms S(J^r) and S(j^~^-) , the approximation taS(e!*) becomes 

,S(^)-Ij<y£) + IsO-£^£), 0<*<*, (32) 

for positive values of ® 9 and 

S(en-jSUj)^SU^f^), -*<*><0, (33) 

for negative values of co. 

Using similar analysis to that embodied in Equations 13 - 18, it 
can be shown that 

Ax . Ax 

Q<o><it , (34) 

for positive values of a>, and 
1 



F( ^f )/G( . J -^ )+ v (e ^ )/GW (^ ) 
Ax hx 



~k <co <0 , (35) 

for negative values of c&. Equations 34 and 35 determine U(d m ) more accurately 
than Equation 18, since they incorporate aliasing between adjacent copies of S in 
the sampling representation for S(el a) ). However, they require sub-pixel shifts of 

— between successively captured images. This corresponds to a factor of 2K 

increase in data acquisition, whereas the resolution is only enhanced by a factor of 
K. 

An alternative approach to improving the approximation in 
Equation 18 by taking into consideration the aliasing between adjacent copies of S 
in the right-hand side of Equation 14, uses an alternating ("checkerboard") mask 
geometry. As mentioned hereinabove, the above discussion is based on use of a 
periodic mask, with a period of By using instead a mask with a period of 2£x, 
one effectively generates two pixel sensitivity functions, gj(x) and g2(*)> where 
gl(x) applies to even-numbered detectors and g2(x) applies to odd-numbered 
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detectors. Specifically, gj(x) corresponds to g(x)m(x) and g2(x) corresponds to 
g(x)m(x+dx), where g(x) is the pixel sensitivity function for the image sensing 
device and m(x) is the overall mask. Unlike the above case where the mask has a 
period of Ax f when the mask has a period of 2 Ax there are effectively two masks 
operating (namely, the mask itself and the mask shifted by Ax)> and consequently 
the even and odd-numbered pixels are processed differently. 

Reference is now made to Figure 11, which is a simplified 
illustration of an alternating ("checkerboard") mask attached to a sensor panel of 
array detectors in an image sensing device, in accordance with a preferred 
embodiment of the present invention* A sensor panel 1100 in an image sensing 
device contains a two-dimensional array of detectors 1110. Detectors 1110 are 
depicted as individual cells within sensor panel 1100. The array of detectors is 
periodic, with periodic spacings Ax and Ay in the horizontal and vertical 
dimensions, respectively. The spacings Ax and Ay correspond to the pixel 
spacings in the sampled digital image obtained from the image sensing device. 

In distinction to Figure 7, Figure 11 illustrates two different 
masking patterns 1120 and 1130 in the form of fine sub-pixel transmission 
gratings that are arranged in an alternating checkerboard pattern to generate a 
doubly periodic mask, with respective periods 2Ax and 2Ay in the horizontal and 
vertical dimensions. In a preferred embodiment of the present invention, the 
doubly periodic mask is attached to sensor panel 1100. The detectors 1110 are 
alternatively masked by the different masking patterns 1120 and 1130. 

When using such a mask, Equation 2 (which is one-dimensional) 
separates into two equations, 

"r Ax 

u k [n]~ ^(x + nAx + k—^g^xydx, 

for even values of n; k = Q, 1, K-I, and (36) 

u k[ n ]~ \u(x + n6x + k—)g 2 (x)dx , 

for odd values of n; k ~ 0 f 1, K~l. (37) 
By capturing twice as many images as previously, namely, 2K images, each 
detector moves over a full period of the mask, namely, 2Ax. Consequently, two 
sequences of the form of Equation 3 are acquired, namely, 

y { [n] = )u(x + n ^)g(x)dx , n = 0 7 1, KN-1 , and (38) 

y,[n] = )u(x + n£~)g(x)dx , n = 0, 1, KN-1 . (39) 
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In turn, using the same analysis embodied in Equations 13 - 18, one arrives at the 
two conditions 



(40) 
(41) 

(42) 
(43) 



for positive values of co, 0 < co < tt, and 

)^jS(jj)G l (-jj) + jS(j— — )G t (-j — — ) , 

K, (e ; ) a -5(; -)G, (-j—) +—S(j ) G 2 (-; ) 

for negative values of £>, -;r < co < 0. These equations can be used to solve for the 
individual copies S(j~) and S(j^~-), and then to recover S(d eo ) from 
Equations 32 and 33. The final result obtained, expressed in terms of U(e! <a ), is 
U(e Jm )* 



Y 2 (en^G 1 (-j^f)-Y l (e J nS 2l[f cG 2 (-j~) 



G 2 (-j —-^mG^-j — -) - G, (-; — ^S^G.X-] — ) 
Ax — Ax Ax — Ar 



Ac 



(44) 



for positive values of co, 0 < oo < tc, and 



— Ax -ZT ^ . 



G z (-J -—)S 2nK G x (-; — ) - G x (-] —)5 2xK G 2 {-] —-) 
Ax — Ax Ax — Ax 



(45) 



for negative values of a, -rz < m < 0. In Equations 44 and 45, the terms S^G(-jn) 

denote the difference 

8*G(-jty = G(-jO) - Gf-/<^~ ^ . (46) 

In distinction to the approach using Equations 34 and 35, whereby 

the improvement in accuracy stems from reducing the sub-pixel sampling shift in 

half, from — to , the approach using Equations 42 and 43 obtains 

K 2.K 

Ax 

improvement in accuracy by acquiring 2K sampled images, each shifted by — * 

Both approaches require collection of sampled data that is 2K as much data as in a 
single image captured by a sensing device. They differ in the types of masks used, 
and in the relative shifts between the multiple images acquired. 

When implemented with full two-dimensional processing, both 
approaches described hereinabove require 4K~ as much data as in a single image 
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captured by a sensing device, in order to enhance image resolution by a factor of 
two in each pixel dimension. 

Geometric Super Resolution vs. Dfffractive Super Resolution 

The multiple image acquisition described above and illustrated in 
Figure 6 is geometric in nature; i.e., it is based on viewing the same spatial 
information several times, such that in each view the object is shifted by a sub- 
pixel amount. It can be implemented by shifting positions of the sensing device 
("temporal multiplexing") or, equivalent by shifting fields of view ("spatial 
multiplexing") using an optical element that replicates views of the object with 
sub-pixel shifts between the replicas. The resolution of a system limited by 
geometry corresponds to the detection sampling system; i.e., the resolution of 
system sensors. 

In distinction, multiple image acquisition can also be diffractive 
in nature. The resolution of a system limited by diffraction corresponds to the 
finest detail that can pass through the system without being distorted, and is 
proportional to the size of the aperture in the optical system. Thus the resolution 
of the human visual system is limited by the extent to which the eyes are open. 
When an observer squints his eyes, the resolution of a scene being viewed 
decreases, since the size of the opening is decreased. 

For example, resolution enhancement can be implemented by an 
assembly of two moving rotated gratings, a first grating attached to the object 
being captured, and a second grating attached to a sensor of the sensing device, 
and moving in a direction opposite to that of the first grating. Such an assembly 
can be used to enhance the resolution of a diffractive system, since it effectively 
increases the size of the aperture in the optical system. The movement of the 
grating attached to the object encodes its spatial information by a Doppler-like 
effect, and allows the information to be transmitted through a restricted aperture of 
an imaging lens. The decoding of the information is obtained from the second 
grating attached to the sensing device, which moves in a direction opposite to that 
of the first grating. Such an assembly of two rotated gratings is practical for 
microscopic applications. 

However, in many imaging systems the object is distant from the 
observer, and attachment of a grating to the object is impractical. When applied to 
scanning systems such as barcode readers, the present invention preferably 
simulates attachment of a moving grating to the object by illuminating the object 
with an illumination having a spatial structure of a grating. The illumination is 
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shifted with time in order to simulate temporal motion of the grating. A specially 
designed diffractive optical element is attached to the illumination source so that a 
, grating pattern appears in the object plane. The grating pattern is moved by phase 
modulating the light source, since linear phase modulations appear as shifts in a 
far field. 



Additional Considerations 

In reading the above description, persons skilled in the art will 
realize that there are many apparent variations that can be applied to the methods 
and systems described. For example, the integral in Equation 3 may be replaced 
by a discrete Gabor transform over it with window function g; namely, 

Similarly, one may use instead a wavelet or Mellin transform, as is well known to 
persons skilled in the art. 

Additionally, the shifts between the multiple images that are 
acquired by the sensing device are not required to be equal. Furthermore, the 
shifts between the multiple images are not required to be horizontally or vertically 
disposed, nor are they required to be disposed in a fixed direction. Wavelet 
transforms and Mellin transforms are particularly advantageous when dealing with 
unequal shifts between acquired images. 

Additionally, the reconstruction algorithm used in the present 
invention, for generating a high resolution image from multiple low resolution 
images, need not be separable. That is, it need not be comprised of horizontal and 
vertical reconstructions. Rather, the reconstruction algorithm used in the present 
invention can be a genuine two-dimensional algorithm. 

In the foregoing specification, the invention has been described 
with reference to specific exemplary embodiments thereof. It will, however, be 
evident that various modifications and changes may be made to the specific 
exemplary embodiments without departing from the broader spirit and scope of 
the invention as set forth in the appended claims. Accordingly, the specification 
and drawings are to be regarded in an illustrative rather than a restrictive sense. 
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CLAIMS 

What is claimed is: 



1. A method for enhancing the resolution of an image sensing 
device, comprising the steps of: 

attaching a mask to a panel of detectors in an image sensing 

device; 

generating multiple fields of view, the multiple fields of view 
being related to one another by sub-pixel shifts; 

acquiring multiple images with the image sensing device from the 
multiple fields of view; and 

combining the multiple images into an enhanced image of higher 
pixel resolution than the pixel resolutions of the multiple images. 

2. The method of claim 1 wherein the image sensing device is a 
scanner. 

3. The method of claim 2 wherein the image sensing device is a 
barcode reader. 

4. The method of claim 1 wherein the image sensing device is a 
CCD camera. 

5. The method of claim 1 wherein the mask is a fine transmission 
grating. 

6. The method of claim 5 wherein the panel of detectors comprises a 
periodic array of detectors, and the mask is periodic with period equal to that of 
the periodic array of detectors. 

7. The method of claim 5 wherein the panel of detectors comprises a 
periodic array of detectors, and the mask is periodic with period twice that of the 
periodic array of detectors. 
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8 * The method of claim 5 further comprising the step of determining 

the number of slits, the positions of the slits and the widths of the slits in the fine 
transmission grating. 

9- The method of claim 8 wherein said determining step determines 

the number of slits, the positions of the slits and the widths of the slits, so that the 
mask eliminates zeroes in a Fourier transform of a function representing a 
sensitivity of a pixel, within a prescribed range of frequencies. 

10. The method of claim 9 wherein the widths of the slits are 

constrained so as not to block more than a prescribed fraction of light energy from 
reaching the detectors. 

11- The method of claim 9 wherein the widths of the slits are 

constrained so as not to be smaller than a prescribed minimum width. 

12. The method of claim 9 wherein said determining step uses a 
weight function specifying relative weights to be placed on values for the widths 
of the slits. 

13. The method of claim 9 wherein the mask is of the form 

m(x)^f d rect(^ r )®S(x-x i ) . 

14. The method of claim 1 wherein said generating step uses natural 
vibrations of a platform supporting the sensing device. 

15. The method of claim 1 wherein said generating step uses at least 
one mirror to shift fields of view of the image sensing device by sub-pixel shifts. 

16. The method of claim 1 wherein said combining step is applied 
separately to rows and columns of pixel data. 



26 



WO 00/59206 PCT/IL00/00201 

17. The method of claim 16 wherein said combining step uses the 
approximation 

G(rj — ) 
Ax 

to determine the rows and columns of pixel data of the enhanced image. 

18. The method of claim 17 wherein said combining step comprises 
applying an inverse Fourier transform to values of U(ei m ). 

19. The method of claim 16 wherein said combining step uses the 
approximation 



Ax Ax 



for positive values of a>, and 



.01 



Q<G><X , 



Ax Ax 



— 7t <a> <0 , 

for negative values of to determine the rows and columns of pixel data of the 
enhanced image. 

20. The method of claim 19 wherein said combining step comprises 

applying an inverse Fourier transform to values of Ufef 6 *). 



21. The method of claim 16 wherein said combining step uses the 

approximation 



U(e J<0 )* 



— Ax — Ax 

Ax Ac 



G z (-J —-)S 2sK G l (-; — ) - G t (-; — )S 2kK G z (-j — ) 
Ax — Ax Ax — Ax 

Ar Ac 



for positive values of <d, 0 < co < it, and 



rr Ax -— Ax 
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for negative values of m, -x < co < 0, and wherein the terms 8yG(-jf2) denote the 
difference 

SvG(-jQ = G(-jO) - G(-j(I2- ¥)) , 
to determine the rows and columns of pixel data of the enhanced ima<*e. 



~* The method of claim 21 wherein said combining step comprises 

applying an inverse Fourier transform to values of U(eJ Q) ). 

23 • The method of claim 1 wherein said combining step uses a Gabor 

transform. 



24. The method of claim 1 wherein said combining step uses a 

wavelet transform. 



25. The method of claim 1 wherein said combining step uses a Mellin 

transform. 



26. A system for enhancing the resolution of an image sensing 
device, comprising: 

an image sensing device comprising a panel of detectors; 
a mask attached to said panel of detectors; 

a motion generator generating multiple fields of view, the 
multiple fields of view being related to one another by sub-pixel shifts; 

image acquisition circuitry housed within said image sensing 
device acquiring multiple images from the multiple fields of view; and 

a combiner combining the multiple images into an enhanced 
image of higher pixel resolution than the pixel resolutions of the multiple images, 

27. The system of claim 26 wherein said image sensing device is a 
scanner. 
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28. The system of claim 27 wherein said image sensing device is a 

barcode reader. 



29. The system of claim 26 wherein said image sensing device is a 

CCD camera* 

30* The system of claim 26 wherein said mask is a fine transmission 

grating. 

31. The system of claim 30 wherein said panel of detectors comprises 
a periodic array of detectors, and said mask is periodic with period equal to that of 
said periodic array of detectors. 

32. The system of claim 30 wherein said panel of detectors comprises 
a periodic array of detectors, and said mask is periodic with period twice that of 
said periodic array of detectors. 

33. The system of claim 30 further comprising a grating generator 
determining the number of slits, the positions of the slits and the widths of the slits 
in the fine transmission grating. 

34. The system of claim 33 wherein said grating generator determines 
the number of slits, the positions of the slits and the widths of the slits, so that said 
mask eliminates zeroes in a Fourier transform of a function representing a 
sensitivity of a pixel. 

35. The system of claim 34 wherein the widths of the slits are 
constrained so as not to block more than a prescribed fraction of light energy from 
reaching the detectors. 

36. The system of claim 34 wherein the widths of the slits are 
constrained so as not to be smaller than a prescribed minimum width. 
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37 * system of claim 34 wherein said grating generator uses a 

weight function specifying relative weights to be placed on values for the widths 
of the slits. 

38 - The system of claim 34 wherein said mask is of the form 

m(x) = l-Y rect(—) ® S(x - x ) . 

39- The system of claim 26 wherein said motion generator uses 
natural vibrations of a platform supporting the sensing device. 

40- The system of claim 26 wherein said motion generator uses at 
least one mirror to shift fields of view of the image sensing device by sub-pixel 
shifts. 

41. The system of claim 26 wherein said combiner is applied 
separately to rows and columns of pixel data. 

42. The system of claim 41 wherein said combiner uses the 
approximation 

G(-J — ) 
Ax 

to determine the rows and columns of pixel data of the enhanced image. 

43. The system of claim 42 wherein said combiner applies an inverse 
Fourier transform to values of t/fe/*). 

44- The system of claim 41 wherein said combiner uses the 

approximation 



2 



Ax Ax 



0 <a> <k , 

for positive values of a>, and 
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1 



Ax Ax 



-K <CO <0 , 

for negative values of co, to determine the rows and columns of pixel data of the 



enhanced image. 



45, The system of claim 44 wherein said combiner comprises 
applying an inverse Fourier transform to values of U(e} m ). 

46. The system of claim 41 wherein said combiner uses the 
approximation 



U{e j( *)* 



tt Ax — Ax 



G 2 (rJ — )$2*kGi(-J—) -Gi(rJ ~ — ) 
Ax — Ax Ax — Ax 



for positive values of co, 0 < a> < 7t, and 
U(e Jm )* 



— Ax Ax 



G z (-j—~)d ^G^-]— )-(?!(-;— )d 2^G z (-j—) 
Ax Ax Ax — Ax 

for negative values of cd } -x < <d < 0, and wherein the terms 5*}G(-]£2) denote the 
difference 

8*GttD) - GUP) - G(-j(f2- ¥)) , 
to detennine the rows and columns of pixel data of the enhanced image, 

47. The system of claim 46 wherein said combiner comprises 
applying an inverse Fourier transform to values of llfei 6 *). 

48. The system of claim 26 wherein said combiner uses a Gabor 
transform. 



49. 

transform. 



The system of claim 26 wherein said combiner uses a wavelet 
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50. The system of claim 26 wherein said combiner uses a Mellin 
transform. 

51. A method for enhancing the resolution of an image sensing 



device, comprising the steps of: 

creating replicas of fields of view using an optical element 
attached to an image sensing device; 

acquiring multiple images with the sensing device from the 
replicas of fields of view; and 

combining the multiple images into an enhanced image of higher 
pixel resolution than the pixel resolutions of the multiple images. 

52. The method of claim 51 wherein the optical element is a 
diffractive optical element. 

53. The method of claim 51 wherein the optical element is a 
reflective optical element. 

54. The method of claim 51 wherein the optical element is a 
combined diffractive and reflective optical element. 

55. The method of claim 51 wherein the optical element produces an 
effect of a grating. 

56. The method of claim 51 wherein the optical element is multi- 
faceted. 

57. The method of claim 51 wherein the replicas of fields of views 
are non-overlapping. 

58. A system for enhancing the resolution of an image sensing 
device, comprising: 

an image sensing device; 
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an optical element attached to said image sensing device, said 
optical element being such as to create replicas of fields of view; 

image acquisition circuitry housed within said image sensing 
device acquiring multiple images from the replicas of fields of view; and 

a combiner combining the multiple images into an enhanced 
image of higher pixel resolution than the pixel resolutions of the multiple images. 

59. The system of claim 58 wherein said optical element is a 
diffractive optical element. 

60. The system of claim 58 wherein said optical element is a 
reflective optical element. 

61. The system of claim 58 wherein said optical element is a 
combined diffractive and reflective optical element, 

62. The system of claim 58 wherein said optical element produces an 
effect of a grating. 

63. The system of claim 58 wherein said optical element is multi- 
faceted. 

64. The system of claim 58 wherein the replicas of fields of view are 
non-overlapping. 
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on and as amended under PCT Article 19 on (if any). 



SXJPPLEMIINTAL DECLARATION (37 CF.R. Section 1.67(b)) 

(complete the following where a sttpplemental declaration is being submitted) 

I ] I hereby declare that the subject .matter of the 

[ ] attached amendment 

[ ] amendment filed on . 

was part of my/our invention and was invented before me filing date of the original 
application, above identified, forsuch invention. 



ACKNOWLEDGMENT OF REVIEW OF PAPERS AND DUTY OF CANDOR 

I hereby slate that I have reviewed and understand the contents of the above-identified 
specification, including the claims, as amended by any amendment referred to above. 

I acknowledge the duty to disclose information, which is material to patentability as delmed in 
37 a Code of Federal Regulations, Section 1.56, 

(also check the following items, if desired) 

I ] and which is material to the examination of this application, namely, information 
where there is a substantial likelihood that a reasonable Examiner would consider it 
important in deciding whether to allow the application to issue as a patent, and 

[J in compliance with this duty, there is attached an information disclosure 
statement, in accordance with 37 C.F*R Section 1.98, 

PRIORITY CLAIM (35 U,S.C. Section I19(a>(d)) 

NOTE: "The claim to priority need be in no special form and may be wade by the attorney or agent if the foreign 

application is referred to in the oath or declaration as required by Section J. 63, The claim for priority and the 
certified copy of (he foreign application specified In 35 U.SC, Section 119(b) must be filed in the case of an 
interference (Section 1,630), when necessary to overcome the date, of a reference relied upon by the examiner, when 
specifically required by the examiner, and in all other situations, before the patent is granted If the claim for 
priority or the certified copy of the foreign application is filed after the date the issue fee Is paid, U must b& 
accompanied by a prMtion requesting entry and by the fen set forth in Section 1.17(f)* if the certified copy is not in 
the English language, a translation need not be fled except in the case of interference; or when ttec&ssary to 
overcome the date of a reference relied upon by the examiner; or when specifically required by the examiner, in 
Which event an English language translation must be filed together with a statement that tte translation of the 
certified copy Is accurate,'* 37 CFJL Section 1.55(a), 

I hereby claim foreign priority benefits under Title 35, United States Code, Section 119(a)-(d) 
of any foreign application^) for patent or inventor's certificate or of any PCT international 
appiication(s) designating at least one country other than the United States of America listed below 
and have also identified below any foreign application(s) for patent or inventor's certificate or any 
PCT international application^) designating at least one country other than the United States of 
America filed by me on the, same subject matter having a filing date before that of the applicants) of 
which priority is claimed. 

(complete (d) or (e)) * 
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(d) [ ] no such applications have been filed. 

(e) £ ] such applications have been filed as follows. 

NOTE: Whore item (c) if entered ab&9£ and lha International Application which designated (he US. itself claimed priority 
chuck item (e) t enter the details below and make the priority claim. 



PRIOR FOREIGN/PCT AFPLICATION(S) FILED WITHIN 12 MONTHS 
(6 MONTHS FOR DESIGN) PRIOR TO THIS APPLICATION 
AND ANY PRIORITY CLAIMS UNDER 35 U.S>C. SECTION II9(a)-(tl) 



COUNTRY (OR 
INDICATE IF 
PCf) 


APPLICATION NUMBER 


DATE OF FILING 
DAY, MONTH, YEAR 


PRIORITY 
CLAIMED 
UNDER 3 5 USC 
119 


PCT 


PCT/IL00/OO201 


30 MaW;h 2000 


fXlYBS f INO 








F IYES r 1NO 








r IYES f 1NO 








r iyes r ino 








r iyes r ino 



CLAIM FOR BENEFIT OF PRIOR U.S. PROVISIONAL APPLICATION^) 

(35 U.S.C. Section 119Ce)) 

I hereby claim the benefit under Title 35, United States Code, Section 1 19(e) of any United 
States provisional application^) listed below: 

PROVISIONAL APPLICATION NUMBER FILING DATE 
_ — ~ 

,/_ — . 

/ 



CLAIM FOR BENEFIT OF EARLIER U.S./PCT APPLICATION^) 
UNDER 35 SECTION 120 

[ ] The claim for the benefit of any such applications are set forth in the attached 

ADDED PAGES TO COMBINED DECLARATION AND POWER OF ATTORNEY 
FOR DIVISIONAL, CONTTNUATTON OR CONTINUATEON-IN-PART (C-I-P) 
APPLICATION. 
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ALL FOREIGN APPLICATION^), IF ANT, FILEX> MORE THAN 12 MONTHS 
(6 MONTHS FOR DESIGN) PRIOR TO THIS XLS, APPLICATION 



NOTE: If the application filed more than 12 months from the filing data of (his application is a POT filing forming the 

basis for this application enteritis the United States as (J J the national #ras&> 0ft (2) a continuation divisional, or 
continuation-in-part, then also complete ADDED PAGES TO COMBINED DECLARATION AND POWER OF 
ATTORNEY FOR DIVISIONAL, CONTINUATION OR C-I-P APPLICATION for benefit of the prior US. or PCT 
application® under 35 U.S.C Section 120. 




POWER OB" ATTORNEY 

I hereby appoint the following practitioner(s) to prosecute this application and transact all 
business in the Patent and Trademark Office connected therewith. 

(list name and registration number) 

JOSEPH H, HAND ELMAN, 26122- JULIAN H- COHEN, ,20302 

JOHN RICHARDS, 31053. WILLIAM R. EVANS-258SS 

RICHARD J, STREIT, 25765, JANET I. CORD, 3333% 

PETER D. GALLOWAY, 278S5 CLIFFORD L MASS 

IAIN C, BAIL LIE, 24090, CYNTHTA R. MILLER, 34g2£ 
RICHARD P. BERG, 2 8145 

(Check the following item, if applicable) 

{ ] I hereby appoint the practitioner(s) associated with the Customer Number provided 
below to prosecute tins application and to transact all business in the Patent and 
Trademark Office connected therewith. 

[ ] Attached, as part of this declaration and power of attorney, is the authorization of the 
above-named practitioners) to accept and follow instructions from my 
representative's). 

NOTE: "Special care should be taken in continuation or divisional applications to ensure that any change of 

corr^pondnnCG address in a prior application is reflected in the continuation or divisional application. For 
example, where a copy of the oath or declaration from the. prior application Is submitted for a continuation or 
divisional application fiUd under 37 CFR 1.53(b) and the copy of the oath or declaration from theprmr 
application designates an old correspondence address, f/re Office may not recognize* in the continuation or 
divisional application, the change of correspondence address made during the prosecution of the prior application. 
Applicant is required to identify the change of correspondence address in the continuation or divMonah application 
to ensure that communications from the Office ore mailed to the current correspondence address, 37 CFK 
L63(d)(4)» Section 601.03, MP.&P-. 7th Ed 
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SEND CORRESPONDENCE TO DIRECT TELEPHONE CALLS TO: 

(Maine and telephone number) 

X^adas-^-Pa^Pty™ Julian H. Cohen 

26_VV.es±j3l-:. t .Street (212) 708-1887 

New York^N.Y. 10023 



(complete the following if applicable) 

Since this filing is a [ ] continuation [ ] divisional there is attached hereto a Change of 
Correspondence Address so that there will be no question as to where the PTO should direct all 
correspondence. 



DECLARATION 

I hereby declare That all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fme or imprisonment, or both, under Section 1001 of Title 18 of the United States Code, and that 
such willful false statements may jeopardize the validity of the application or any patent issued 
thereon. 
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SIGNATURE (S) 

HOTE: Carefully indicate the family for last) name, as it should appear on the filing receipt and all other document. 

NOTE: Each inventor must be identified by full name, including the family name, and at least &ne given name without 
abbreviation together with any ether given name or initial, and*by his/her residence, post office address and 
country qf citizenship. 37 C/\JL Section 1.6$(a)(B)* » 

NOTE: Inventors may execute separate declarations/oaths provided f.aeh declaration/oath sets forth el! the inventors. . 

Section 2. €3 (a) (3) requires that a declaration/oath !»ter alia, Identify each inventor and prohibits the execution of 
separate declarations/oaths -which each sets forth only the name of the executing Inventor. tf2 fed. Jle$ 9 J3j3j t 
SBj42 l October IQ, 199% 



Full name of sole or first inventor 

vm 



u X&tyen name) 



Inventors signature (xl 



J?a7ttily*{Or JLast Name) 



Date (x) fyfllj/fl'j Country of Citizenship 
Residence Patach T1kya flL^ 7 ; ~ X 



'top- 

Israel 



Post Office Address IDEUMAN ST, 49S50 Petacb Tikva Israel 



Full name of second joint inventor, if any 

(^IvmNatne) JMiddle initial or Name) ^IFmnty^rjLctst jvamej 



Dm 

Inventor's signature (x\ 



l^ata M A** I i&l OA Country of atizenkhip ' 1 y_Israe1 
Residence Rosh^tiaavin (1L) - 1 




Post Office Address M^vmnn c» 1, Aftssn Rnch Haayin Israel. 



Full name of third joint inventor, if any 

f&Iven JVame) (Middle Initial or JNaJne) k amity (Or i>asi isame) 

Inventor's signature OQ — — — 

Date OP * Country of Citizenship TSKAJ^' — — 

•ftestdence - • — 

Post Office Address - - — — — 
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(check proper boxfes) for any of the following added pagefs) 
that form a part of this declaration) 

[ ] Signature for fourth and subsequent joint inventors. Number of pages added 

* * * 

[ ] Signature by admin istrator(trix) 3 executor(trifc) or legal representative for deceased or 
incapacitated inventor, Mtrnber of pages added 



[ ] Signature for inventor who refuses to sign or cannot be reached by person authorized under 
37 CF.R. Section L47. NumSer of pages added 



[ ] Added page for signature by one joint inventor on behalf of deceased inventor(s) where legal 
representative cannot be appointed in time, (37 C.F.R. Section 1.47) 



[ ] Added pages to combined declaration and power of attorney for divisional, continuation, or 
continuation-in-part (CM-P) application* 

[ ] Number of pages added _ , . 



* * # 



[ ] Authorization of practltioner(s) to accept and follow instructions from representative. 

(If no further pages form a part of this Declaration, 
then end ihte Declaration with this page and check the following item) 

[yj This declaration ends with this page. 
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